1. Field of the Invention
The invention relates to real time velocity measurements of particles flowing in a venue. In one embodiment, the present invention relates to the in vivo measurement of the velocity of particles such as rolling leukocytes. The methods of the invention are also suitable for tracking the movement of larger particles or objects such as vehicles in the flow of traffic. An apparatus suited for using the method of the invention is also provided.
2. Description of the Background
It is presently impractical to implement a procedure for real time tracking of the velocity and stopping patterns of particles within a circulating venue. For purposes of describing this field and invention, by xe2x80x9cparticlexe2x80x9d is meant a discrete object of any size from microscopic to the size of vehicles. By xe2x80x9cvenuexe2x80x9d is meant the environment in which particles move. Circulating venues range from the size of capillaries to pipes to highways. The principles to be taught herein are applicable to any particle in any circulating venue.
Presently available technology is labor intensive, and incapable of tracking multiple venues at one time automatically and providing real time information. Such technology has application to a variety of fields, including tracking cells in the blood flow and circulation of motor vehicles in specific venues. In the latter case, the availability of such technology would permit tracking and modifying the circulation of, for example, motor vehicles during peak hours of traffic or in cases of accidents, which normally produce extended delays in circulation.
In the case of circulating cells, for example, tissue damage or infection initiates a complex chain of events in which certain substances such as cytokines and chemoattractants are released in the interstitium and diffuse towards small veins (venules). These substances activate venular endothelial cells (the cells lining the blood vessel wall) to produce and express adhesion molecules. Leukocytes in the blood then transiently attach to the endothelial cells by binding to these adhesion molecules. These attachments cause the velocity of the leukocytes to decrease relative to the plasma and red blood cells, and the leukocytes appear to xe2x80x9crollxe2x80x9d along the venular wall. Additional chemo-attractants and released mediators of inflammation induce the expression of other adhesion molecules that form stronger leukocyte-endothelial interactions. The velocity of the leukocyte approaches zero as it firmly adheres or xe2x80x9csticksxe2x80x9d to the vascular wall. Subsequently, these stuck leukocytes migrate through the vascular wall into the tissue where they participate in phagocytosis and the inflammatory process.
Many of the substances released by the leukocytes destroy invading bacteria and also damage host tissue. This is often a major clinical problem in inflammatory conditions. Indeed, an uncontrolled leukocyte activation may in itself be the most damaging and deleterious aspect of a disease because normal tissue may be damaged or destroyed. Continuous measurement of the velocity of leukocytes in multiple vessels during the early stages of the inflammatory response is essential to understanding and eventually controlling leukocyte function. This is also true about the continuous measurement of the velocity of other particles, such as motor vehicles, in multiple circulating venues, for understanding and modifying traffic patters. Yet, current methodology in this area is time consuming and severely limits the quantity of data that can be collected.
In vivo television microscopy may be used to view the first signs of particle behavior, e.g. in the case of leukocyte activation, which is manifested by xe2x80x9crollingxe2x80x9d and then xe2x80x9cstickingxe2x80x9d of the leukocytes to the venular wall. To be able to quantify this early phase of the inflammatory response by measuring changes in leukocyte velocity is extremely important. Currently available methods of quantifying these velocities in vivo, however, are labor intensive. For example, in a manual method of measuring leukocyte velocity or adhesion, videotape replay with a frame by frame analysis is used to count, or determine the velocity of individual leukocytes in a particular section of a blood vessel. This method of data collection is time consuming and laborious and restricts data collection to less than a minute for a single segment of each single vessel, and at only a few time points during the course of any extended test. Furthermore, this manual method and those utilizing computer-assisted systems have the ability of measuring leukocyte velocities in only a small section of one blood vessel at only a few time points per test.
In addition, because computer-assisted and manual methods are time-consuming and subjective in the choice of leukocytes, many leukocytes are often missed. In addition, this, by itself, makes the prior art methods inadequate for correlating, for example, adhesion protein expression sites within the wall of a particular blood vessel with leukocyte velocity.
This is also true for other systems of particles and objects. Similarly important is the quantification of traffic patterns by tracking changes in vehicular velocities. Because current methods are generally performed by off-line analysis of videotape, it is difficult to know during the time lapse on an on-going test how it is progressing.
Commercially available general-purpose image analysis software packages, such as Optimas BioScan, ImagePro Plus, and Inspector, have scripting or macro capabilities and therefore have some automatic analysis capability. These programs, however, require human intervention and decision making at multiple points in the analysis and cannot implement advanced analysis processes. Moreover, existing software is not fast enough for near real-time measurements of the velocities of particles such as leukocytes. Some of the commercial systems are used to assess leukocyte-endothelial interactions in vitro, where flow parameters of the fluid surrounding the leukocytes may be carefully controlled and monitored. Systems designed to work under these conditions do not work well in real time (in vivo) because of the visual heterogeneity in the tissue.
Over the past decade, several computer systems have been developed to assist with the measurement of rolling leukocyte velocities in vivo. While these systems are somewhat helpful, they utilize playback of a videotaped experiment, require the user to interactively select the white blood cells to be measured in sequential video frames, and record data from only a few minutes per test.
Recently, Sato et al. disclosed a technique for automatically tracking individual leukocytes in image sequences. See, Sato Y., et al., xe2x80x9cAutomatic Extraction and Measurement of Leukocyte Motion in Microvessels Using Spatiotemporal Image Analysisxe2x80x9d, IEEE Trans. Biomed Eng. 44(4): 225-236 (1997). The Sato et al. method consists of a long series of computationally expensive steps and requires several user specified parameters. It does not appear that the system has feasible capability of real-time or near real time analysis.
Accordingly, it would be desirable to provide a system for the rapid, accurate and automatic measurement of tracking the velocities of particles, such as leukocyte rolling velocities and motor vehicles, in circulating velocities in real time. It also would be desirable that the system identify particle stoppages, e.g. adhering leukocytes, in multiple venues, e.g. vessels, and determine the period of time during which each stopped particle and object, e.g. adhering leukocyte, has zero or near zero velocity.
This invention, thus, relates to a method of tracking particles, generally comprising the generation of a time-varying sequence of images of a circulating venue containing a flow of particles, determining spatio-temporal intensity gradients within each of the images of the circulating venues, and identifying particles within the images of the circulating venue based upon the thus determined spatio-temporal intensity gradients. The particles or objects, e.g. leukocytes, within the images are matched with the corresponding identical particles, e.g. leukocytes, from other images using a heuristic technique. The method may additionally comprise other steps and/or execute some of the described steps in several stages. For purposes of clarity, the present invention will be generally described for the tracking of leukocytes and the determination of leukocyte rolling velocities. However, the present technology is applicable to any system of particles which circulate in one direction, such as street vehicular traffic, and the like.
For example, one preferred embodiment of the method of the invention comprises generating a time-varying sequence of images of a blood vessel containing a flow of blood, determining spatio-temporal gradients within each of the images of the blood vessel, identifying leukocytes within the images of the blood vessel based upon the spatio-temporal gradients, matching leukocytes within the images with the corresponding identical leukocytes from other images, determining an amount of translation of the matched leukocytes from one image to another, and determining movement characteristics of the leukocytes based upon the amount of translation.
In an exemplary embodiment, the time-varying image of a circulating venue containing a flow of particles or objects is generated by imaging the circulating venue using a video camera, selecting a rectangular area of interest within the image representative of a portion of the blood vessel, and then capturing sequential images of the blood vessel using a digital frame grabber. The spatio-temporal intensity gradients within each of the images of the blood vessel may be generated by subtracting an image from a previous image and applying a convolution mask to the resulting subtracted image to determine spatio-temporal derivatives. The spatio-temporal gradients may also be generated by convolving the image sequence with a 3D Sobel operator and representing the result as a 4-D data set, determining embedding angles for the 4-D data set, and determining spatio-temporal derivatives based upon the embedding angles. The leukocytes are then identified within the images by comparing the images to predetermined thresholds. The leukocytes may also be identified by performing a multifeature texture-based segmentation on the spatio-temporal gradients or by applying a multi-layer feed-forward neural network segmentation on the spatio-temporal gradients. The leukocytes may then be matched with corresponding identical leukocytes from other images by using a heuristic segmentation equation or, alternatively, a multi-layer feed-forward neural network segmentation or a correlation operator. The amount of translation of the matched leukocytes from one image to another may be determined, for instance, by calculating a center of gravity for each leukocyte and then calculating a Euclidean translation distance between the centers of gravity of the same leukocyte as it appears in separate images. The leukocyte velocity may be computed by dividing the translation distance by the time differential between the frames. The first five statistical moments of frequency distribution of the velocity may also be determined and used to characterize the immunological state of the organism. It has unexpectedly been found that the first three moments of frequency distribution of the velocity are sufficient to determine a useful approximation.
This invention also relates to an apparatus for practicing the above described method. The apparatus of this invention is capable of automatically tracking particle flow in real time, and comprises a video source providing a time-varying image of a circulating venue containing a flow of particles; an image capture unit capturing a sequence of images of the time-varying image; a processor identifying and tracking particles within the sequence of images of the circulating venue, said processor including one or more of a neural network processor, a heuristic processor, and a template-based correlation matching processor for tracking the particle.
The invention having now been generally described, other embodiments and objects relating to its functioning will now be described in experimental terms with reference to the accompanying drawings. It is well known to those skilled in the art that variations in this invention may be readily made. Therefore, such variations are considered to be within the scope and spirit of this invention.